Adaptation of Learning Rule Parameters Using a Meta Neural Network

This paper proposes an application independent method of automating learning rule parameter selection using a form of supervisor neural network, known as a Meta Neural Network, to alter the value of a learning rule parameter during training. The Meta Neural Network is trained using data generated by observing the training of a neural network and recording the effects of the selection of various parameter values. Experiments are undertaken to see how this method performs by using it to adapt a global parameter of the RPROP and Quickpropagation learning rules.